Skip to content

Return comparable LinkedHashMap in valueForCalcite() of ExprTupleValue#4629

Merged
qianheng-aws merged 2 commits intoopensearch-project:mainfrom
LantaoJin:pr/issues/4339
Oct 23, 2025
Merged

Return comparable LinkedHashMap in valueForCalcite() of ExprTupleValue#4629
qianheng-aws merged 2 commits intoopensearch-project:mainfrom
LantaoJin:pr/issues/4339

Conversation

@LantaoJin
Copy link
Member

@LantaoJin LantaoJin commented Oct 22, 2025 •
edited
Loading

Description

Fix class java.util.LinkedHashMap cannot be cast to class java.lang.Comparable in dedup struct field

Related Issues

Resolves #4339

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@LantaoJin LantaoJin added bug Something isn't working backport 2.19-dev labels Oct 22, 2025
@LantaoJin LantaoJin marked this pull request as ready for review October 22, 2025 09:37
qianheng-aws
qianheng-aws reviewed Oct 22, 2025
View reviewed changes
core/src/main/java/org/opensearch/sql/data/utils/ComparableLinkedHashMap.java Outdated
V thisFirstValue = this.values().iterator().next();
V otherFirstValue = other.values().iterator().next();

if (thisFirstValue instanceof Comparable) {
Copy link
Collaborator

@qianheng-aws qianheng-aws Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only comparing the first element?

Copy link
Member Author

@LantaoJin LantaoJin Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to compare recursively

@LantaoJin
change to compare recursively
1cb6212 
Signed-off-by: Lantao Jin <ltjin@amazon.com>
yuancu
yuancu approved these changes Oct 23, 2025
View reviewed changes
core/src/test/java/org/opensearch/sql/utils/ComparableLinkedHashMapTest.java
Comment on lines +51 to +62
public void testDifferentFirstValue() {
ComparableLinkedHashMap<String, Integer> map1 = new ComparableLinkedHashMap<>();
map1.put("a", 1);
map1.put("b", 2);

ComparableLinkedHashMap<String, Integer> map2 = new ComparableLinkedHashMap<>();
map2.put("c", 3);
map2.put("d", 2);

assertTrue(map1.compareTo(map2) < 0);
assertTrue(map2.compareTo(map1) > 0);
}
Copy link
Collaborator

@yuancu yuancu Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the keys of ExprTupleValues be ignored? Or are they guaranteed to come with the same order?

@qianheng-aws qianheng-aws merged commit 80fb6b7 into opensearch-project:main Oct 23, 2025
51 of 52 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 23, 2025
@github-actions
Return comparable LinkedHashMap in valueForCalcite() of ExprTupleVa…
537a9e9 
…lue (#4629)

* Return comparable LinkedHashMap in valueForCalcite() of ExprTupleValue

Signed-off-by: Lantao Jin <ltjin@amazon.com>

* change to compare recursively

Signed-off-by: Lantao Jin <ltjin@amazon.com>

---------

Signed-off-by: Lantao Jin <ltjin@amazon.com>
(cherry picked from commit 80fb6b7)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Oct 23, 2025
Merged
@LantaoJin LantaoJin mentioned this pull request Oct 23, 2025
Merged
8 tasks
@LantaoJin
Copy link
Member Author

LantaoJin commented Oct 23, 2025 •
edited
Loading

@qianheng-aws I filed a followup PR. The current ComparableLinkedHashMap only compares the values. Change it to compare key than value.

LantaoJin pushed a commit that referenced this pull request Oct 24, 2025
@opensearch-trigger-bot @github-actions
Return comparable LinkedHashMap in valueForCalcite() of ExprTupleVa…
c865a5a 
…lue (#4629) (#4647)

* Return comparable LinkedHashMap in valueForCalcite() of ExprTupleValue



* change to compare recursively



---------


(cherry picked from commit 80fb6b7)

Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
asifabashar added a commit to asifabashar/sql that referenced this pull request Oct 28, 2025
@asifabashar
Merge branch 'default-main'
e7775d3 
* default-main: (34 commits)
  Enhance dynamic source clause to support only metadata filters (opensearch-project#4554)
  Make nested alias type support referring to outer context (opensearch-project#4673)
  Update big5 ppl queries and check plans (opensearch-project#4668)
  Support push down sort after limit (opensearch-project#4657)
  Use table scan rowType in filter pushdown could fix rename issue (opensearch-project#4670)
  Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621)
  Fix bin nested fields issue (opensearch-project#4606)
  Add `per_minute`, `per_hour`, `per_day` function support (opensearch-project#4531)
  Pushdown sort aggregate metrics (opensearch-project#4603)
  Followup: Change ComparableLinkedHashMap to compare Key than Value (opensearch-project#4648)
  Mitigate the CI failure caused by 500 Internal Server Error (opensearch-project#4646)
  Allow renaming group-by fields to existing field names (opensearch-project#4586)
  Publish internal modules separately for downstream reuse (opensearch-project#4484)
  Revert "Update grammar files and developer guide (opensearch-project#4301)" (opensearch-project#4643)
  Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions (opensearch-project#4599)
  Replace all dots in fields of table scan's PhysType (opensearch-project#4633)
  Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue (opensearch-project#4629)
  Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT (opensearch-project#4623)
  Pushdown case function in aggregations as range queries (opensearch-project#4400)
  Update GEOIP function to support IP types as input (opensearch-project#4613)
  ...

# Conflicts:
#	docs/user/ppl/functions/conversion.rst
asifabashar added a commit to asifabashar/sql that referenced this pull request Oct 28, 2025
@asifabashar
Merge branch 'default-main'
42d705a 
* default-main: (34 commits)
  Enhance dynamic source clause to support only metadata filters (opensearch-project#4554)
  Make nested alias type support referring to outer context (opensearch-project#4673)
  Update big5 ppl queries and check plans (opensearch-project#4668)
  Support push down sort after limit (opensearch-project#4657)
  Use table scan rowType in filter pushdown could fix rename issue (opensearch-project#4670)
  Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621)
  Fix bin nested fields issue (opensearch-project#4606)
  Add `per_minute`, `per_hour`, `per_day` function support (opensearch-project#4531)
  Pushdown sort aggregate metrics (opensearch-project#4603)
  Followup: Change ComparableLinkedHashMap to compare Key than Value (opensearch-project#4648)
  Mitigate the CI failure caused by 500 Internal Server Error (opensearch-project#4646)
  Allow renaming group-by fields to existing field names (opensearch-project#4586)
  Publish internal modules separately for downstream reuse (opensearch-project#4484)
  Revert "Update grammar files and developer guide (opensearch-project#4301)" (opensearch-project#4643)
  Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions (opensearch-project#4599)
  Replace all dots in fields of table scan's PhysType (opensearch-project#4633)
  Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue (opensearch-project#4629)
  Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT (opensearch-project#4623)
  Pushdown case function in aggregations as range queries (opensearch-project#4400)
  Update GEOIP function to support IP types as input (opensearch-project#4613)
  ...

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>
expani pushed a commit to vinaykpud/sql that referenced this pull request Nov 4, 2025
@LantaoJin @expani
Return comparable LinkedHashMap in valueForCalcite() of ExprTupleVa…
e6edd44 
…lue (opensearch-project#4629)

* Return comparable LinkedHashMap in valueForCalcite() of ExprTupleValue

Signed-off-by: Lantao Jin <ltjin@amazon.com>

* change to compare recursively

Signed-off-by: Lantao Jin <ltjin@amazon.com>

---------

Signed-off-by: Lantao Jin <ltjin@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yuancu yuancu yuancu approved these changes

@qianheng-aws qianheng-aws qianheng-aws approved these changes

@ps48 ps48 Awaiting requested review from ps48 ps48 is a code owner

@kavithacm kavithacm Awaiting requested review from kavithacm kavithacm is a code owner

@derek-ho derek-ho Awaiting requested review from derek-ho derek-ho is a code owner

@joshuali925 joshuali925 Awaiting requested review from joshuali925 joshuali925 is a code owner

@dai-chen dai-chen Awaiting requested review from dai-chen dai-chen is a code owner

@YANG-DB YANG-DB Awaiting requested review from YANG-DB YANG-DB is a code owner

@mengweieric mengweieric Awaiting requested review from mengweieric mengweieric is a code owner

@vamsimanohar vamsimanohar Awaiting requested review from vamsimanohar vamsimanohar is a code owner

@Swiddis Swiddis Awaiting requested review from Swiddis Swiddis is a code owner

@penghuo penghuo Awaiting requested review from penghuo penghuo is a code owner

@seankao-az seankao-az Awaiting requested review from seankao-az seankao-az is a code owner

@MaxKsyunz MaxKsyunz Awaiting requested review from MaxKsyunz MaxKsyunz is a code owner

@Yury-Fridlyand Yury-Fridlyand Awaiting requested review from Yury-Fridlyand Yury-Fridlyand is a code owner

@anirudha anirudha Awaiting requested review from anirudha anirudha is a code owner

@forestmvey forestmvey Awaiting requested review from forestmvey forestmvey is a code owner

@acarbonetto acarbonetto Awaiting requested review from acarbonetto acarbonetto is a code owner

@GumpacG GumpacG Awaiting requested review from GumpacG GumpacG is a code owner

@ykmr1224 ykmr1224 Awaiting requested review from ykmr1224 ykmr1224 is a code owner

@noCharger noCharger Awaiting requested review from noCharger noCharger is a code owner

@RyanL1997 RyanL1997 Awaiting requested review from RyanL1997 RyanL1997 is a code owner

Assignees

No one assigned

Labels

backport 2.19-dev bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[BUG] dedup raise 500 error when object attribute is specified

3 participants

@LantaoJin @yuancu @qianheng-aws